Performing Realistic Workout Activity Recognition on Consumer Smartphones

Smartphones have become an essential part of our lives. Especially its computing power and its current specifications make a modern smartphone a powerful device for human activity recognition tasks. Equipped with various integrated sensors, a modern smartphone can be leveraged for lots of smart applications. We already investigated the possibility of using an unmodified commercial smartphone to recognize eight strength-based exercises. App-based workouts have become popular in the last few years. The advantage of using a mobile device is that you can practice anywhere at anytime. In our previous work, we proved the possibility of turning a commercial smartphone into an active sonar device to leverage the echo reflected from exercising movement close to the device. By conducting a test study with 14 participants, we showed the first results for cross person evaluation and the generalization ability of our inference models on disjoint participants. In this work, we extended another model to further improve the model generalizability and provided a thorough comparison of our proposed system to other existing state-of-the-art approaches. Finally, a concept of counting the repetitions is also provided in this study as a parallel task to classification

ExerTrack - Towards Smart Surfaces to Track Exercises

The concept of the quantified self has gained popularity in recent years with the hype of miniaturized gadgets to monitor vital fitness levels. Smartwatches or smartphone apps and other fitness trackers are overwhelming the market. Most aerobic exercises such as walking, running, or cycling can be accurately recognized using wearable devices. However whole-body exercises such as push-ups, bridges, and sit-ups are performed on the ground and thus cannot be precisely recognized by wearing only one accelerometer. Thus, a floor-based approach is preferred for recognizing whole-body activities. Computer vision techniques on image data also report high recognition accuracy; however, the presence of a camera tends to raise privacy issues in public areas. Therefore, we focus on combining the advantages of ubiquitous proximity-sensing with non-optical sensors to preserve privacy in public areas and maintain low computation cost with a sparse sensor implementation. Our solution is the ExerTrack, an off-the-shelf sports mat equipped with eight sparsely distributed capacitive proximity sensors to recognize eight whole-body fitness exercises with a user-independent recognition accuracy of 93.5 % and a user-dependent recognition accuracy of 95.1 % based on a test study with 9 participants each performing 2 full sessions. We adopt a template-based approach to count repetitions and reach a user-independent counting accuracy of 93.6 %. The final model can run on a Raspberry Pi 3 in real time. This work includes data-processing of our proposed system and model selection to improve the recognition accuracy and data augmentation technique to regularize the network

Fitness Activity Recognition on Smartphones Using Doppler Measurements

Quantified Self has seen an increased interest in recent years, with devices including smartwatches, smartphones, or other wearables that allow you to monitor your fitness level. This is often combined with mobile apps that use gamification aspects to motivate the user to perform fitness activities, or increase the amount of sports exercise. Thus far, most applications rely on accelerometers or gyroscopes that are integrated into the devices. They have to be worn on the body to track activities. In this work, we investigated the use of a speaker and a microphone that are integrated into a smartphone to track exercises performed close to it. We combined active sonar and Doppler signal analysis in the ultrasound spectrum that is not perceivable by humans. We wanted to measure the body weight exercises bicycles, toe touches, and squats, as these consist of challenging radial movements towards the measuring device. We have tested several classification methods, ranging from support vector machines to convolutional neural networks. We achieved an accuracy of 88% for bicycles, 97% for toe-touches and 91% for squats on our test set

Monolayer hydrophilic MoS2 with strong charge trapping for atomically thin neuromorphic vision systems

Effective control of electrical and optoelectronic properties of two-dimensional layered materials, one of the key requirements for applications in advanced optoelectronics with multiple functions, has been hindered by the difficulty of elemental doping, which is commonly utilized in Si technology. In this study, we proposed a new method to synthesize hydrophilic MoS2 monolayers through covalently introducing hydroxyl groups during their growth process. These hydroxyl groups exhibit a strong capability of charge trapping, and thus the hydrophilic MoS2 monolayers achieve excellent electrical, optical, and memory properties. Optical memory transistors, made from a single component of monolayer hydrophilic MoS2, exhibit not only excellent light-dependent and time-dependent photoelectric performance, but also good photo-responsive memory characteristics with over multi-bit storage and more than 104 switching ratios. Atomically thin neuromorphic vision systems (with a concept of proof of 10 × 10 neuromorphic visual image) are manufactured from arrays of hydrophilic MoS2 optical memory transistors, showing high quality image sensing and memory functions with a high color resolution. These results proved our new concepts to realize image memorization and simplify the pixel matrix preparation process, which is a significant step toward the development of future artificial visual systems

Engineering zinc oxide hybrid selenium nanoparticles for synergetic anti-tuberculosis treatment by combining Mycobacterium tuberculosis killings and host cell immunological inhibition

IntroductionAs a deadly disease induced by Mycobacterium tuberculosis (Mtb), tuberculosis remains one of the top killers among infectious diseases. The low intracellular Mtb killing efficiency of current antibiotics introduced the long duration anti-TB therapy in clinic with strong side effects and increased drug-resistant mutants. Therefore, the exploration of novel anti-TB agents with potent anti-TB efficiency becomes one of the most urgent issues for TB therapies. MethodsHere, we firstly introduced a novel method for the preparation of zinc oxide-selenium nanoparticles (ZnO-Se NPs) by the hybridization of zinc oxide and selenium to combine the anti-TB activities of zinc oxide nanoparticles and selenium nanoparticles. We characterized the ZnO-Se NPs by dynamic laser light scattering and transmission electron microscopy, and then tested the inhibition effects of ZnO-Se NPs on extracellular Mtb by colony-forming units (CFU) counting, bacterial ATP analysis, bacterial membrane potential analysis and scanning electron microscopy imaging. We also analyzed the effects of ZnO-Se NPs on the ROS production, mitochondrial membrane potential, apoptosis, autophagy, polarization and PI3K/Akt/mTOR signaling pathway of Mtb infected THP-1 macrophages. At last, we also tested the effects of ZnO-Se NPs on intracellular Mtb in THP-1 cells by colony-forming units (CFU) counting. ResultsThe obtained spherical core-shell ZnO-Se NPs with average diameters of 90 nm showed strong killing effects against extracellular Mtb, including BCG and the virulent H37Rv, by disrupting the ATP production, increasing the intracellular ROS level and destroying the membrane structures. More importantly, ZnO-Se NPs could also inhibit intracellular Mtb growth by promoting M1 polarization to increase the production of antiseptic nitric oxide and also promote apoptosis and autophagy of Mtb infected macrophages by increasing the intracellular ROS, disrupting mitochondria membrane potential and inhibiting PI3K/Akt/mTOR signaling pathway. DiscussionThese ZnO-Se NPs with synergetic anti-TB efficiency by combining the Mtb killing effects and host cell immunological inhibition effects were expected to serve as novel anti-TB agents for the development of more effective anti-TB strategy

Sensor Applications for Human Activity Recognition in Smart Environments

Human activity recognition (HAR) is the automated recognition of individual or group activities from sensor inputs. It deals with a wide range of application areas, such as for health care, assisting technologies, quantified-self and safety applications. HAR is the key to build human-centred applications and enables users to seamlessly and naturally interact with each other or with a smart environment. A smart environment is an instrumented room or space equipped with sensors and actuators to perceive the physical state or human activities within this space. The diversity of sensors makes it difficult to use the appropriate sensor to build specific applications. This work aims at presenting sensor-driven applications for human activity recognition in smart environments by using novel sensing categories beyond the existing sensor technologies commonly applied to these tasks. The intention is to improve the interaction for various sub-fields of human activities. Each application addresses the difficulties following the typical process pipeline for designing a smart environment application. At first, I survey most prominent research works with focus on sensor-driven categorization in the research domain of HAR to identify possible research gaps to position my work. I identify two use-cases: quantified-self and smart home applications. Quantified-self aims at self-tracking and self-knowledge through numbers. Common sensor technology for daily tracking of various aerobic endurance training activities, such as walking, running or cycling are based on acceleration data with wearable. However, more stationary exercises, such as strength-based training or stretching are also important for a healthy life-style, as they improve body coordination and balance. These exercises are not well tracked by wearing only a single wearable sensor, as these activities rely on coordinated movement of the entire body. I leverage two sensing categories to design two portable mobile applications for remote sensing of these more stationary exercises of physical workout. Sensor-driven applications for smart home domain aim at building systems to make the life of the occupants safer and more convenient. In this thesis, I target at stationary applications to be integrated into the environment to allow a more natural interaction between the occupant and the smart environment. I propose two possible solutions to achieve this task. The first system is a surface acoustic based system which provides a sparse sensor setup to detect a basic set of activities of daily living including the investigation of minimalist sensor arrangement. The second application is a tag-free indoor positioning system. Indoor localization aims at providing location information to build intelligent services for smart homes. Accurate indoor position offers the basic context for high-level reasoning system to achieve more complex contexts. The floor-based localization system using electrostatic sensors is scalable to different room geometries due to its layout and modular composition. Finally, privacy with non-visual input is the main aspect for applications proposed in this thesis. In addition, this thesis addresses the issue of adaptivity from prototypes towards real-world applications. I identify the issues of data sparsity in the training data and data diversity in the real-world data. In order to solve the issue of data sparsity, I demonstrate the data augmentation strategy to be applied on time series to increase the amount of training data by generating synthetic data. Towards mitigating the inherent difference of the development dataset and the real-world scenarios, I further investigate several approaches including metric-based learning and fine-tuning. I explore these methods to finetune the trained model on limited amount of individual data with and without retrain the pre-trained inference model. Finally some examples are stated as how to deploy the offline model to online processing device with limited hardware resources

Biometric Recognition in 3D Medical Images: A Survey

This work explores the potential applications of biometric recognition in 3D medical imaging data. We investigate various 3D imaging techniques commonly used in the medical domain, including 3D ultrasound imaging, magnetic resonance imaging (MRI), computer tomography (CT) scans, and 3D near-infrared (NIR) imaging. For each technique, we provide an overview of its working principle and discuss the advantages of integrating biometrics into 3D medical imaging data. Major advantage of using biometrics in this context is motivated by the research that using biometrics could not only increase data security but, more importantly, decrease the mix-up errors in patient’s medical data and thus improving patient safety and patient care. Our analysis uncovers certain weaknesses in current algorithms and limitations in existing research. Possible reasons include insufficient data availability, the under-utilization of deep-learning-based approaches to enhance accuracy and performance, and the absence of standardized benchmarking databases to support research. Our survey frames existing works and lead to practical recommendations and motivates efforts to improve the current state of research. Beyond exploring the utilization of biometrics in 3D medical imaging data, our study touches on further potential interactions between them, such as extracting health information from biometric captures. This work is thus the first work to survey and present an overview on works proposing the use of medical images for biometric recognition

Face morphing attacks and face image quality: The effect of morphing and the unsupervised attack detection by quality

Morphing attacks are a form of presentation attacks that gathered increasing attention in recent years. A morphed image can be successfully verified to multiple identities. This operation, therefore, poses serious security issues related to the ability of a travel or identity document to be verified to belong to multiple persons. Previous studies touched on the issue of the quality of morphing attack images, however, with the main goal of quantitatively proofing the realistic appearance of the produced morphing attacks. The authors theorise that the morphing processes might have an effect on both, the perceptual image quality and the image utility in face recognition (FR) when compared to bona fide samples. Towards investigating this theory, this work provides an extensive analysis of the effect of morphing on face image quality, including both general image quality measures and face image utility measures. This analysis is not limited to a single morphing technique but rather looks at six different morphing techniques and five different data sources using ten different quality measures. This analysis reveals consistent separability between the quality scores of morphing attack and bona fide samples measured by certain quality measures. The authors’ study goes further to build on this effect and investigate the possibility of performing unsupervised morphing attack detection (MAD) based on quality scores. The authors’ study looks into intra- and inter-dataset detectability to evaluate the generalisability of such a detection concept on different morphing techniques and bona fide sources. The authors’ final results point out that a set of quality measures, such as MagFace and CNNIQA, can be used to perform unsupervised and generalised MAD with a correct classification accuracy of over 70%